Panoramic Video Salient Object Detection with Ambisonic Audio Guidance

Video salient object detection (VSOD), as a fundamental computer vision problem, has been extensively discussed in the last decade. However, all existing works focus on addressing VSOD problem 2D scenarios. With rapid development of VR devices, panoramic videos have promising alternative to provide immersive feelings real world. In this paper, we aim tackle video for videos, with their corresponding ambisonic audios. A multimodal fusion module equipped two pseudo-siamese audio-visual context (ACF) blocks is proposed effectively conduct interaction. The ACF block spherical positional encoding enables 3D capture spatial correspondence between pixels and sound sources from equirectangular frames Experimental results verify effectiveness our components demonstrate that method achieves state-of-the-art performance ASOD60K dataset.